Add Unit and Functional test CI pipeline - #4506
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a new Azure DevOps pipeline (sqlclient-ci-unit) that runs the SqlClient Unit and Functional test suites in Package reference mode, consuming the exact NuGet artifacts produced by the triggering sqlclient-ci-package run. This follows the same downstream-consume-upstream-artifacts pattern introduced by the stacked PR (#4499) and extends it to broad Unit/Functional coverage across OS/TFM/SNI combinations.
Changes:
- Introduces a package-triggered pipeline definition that runs only on completion of
sqlclient-ci-package(no PR/CI triggers). - Defines a 3-stage OS matrix (Windows/Linux/macOS) with the intended TFM and SNI coverage.
- Adds a reusable job template that aligns source to the upstream commit, downloads/stages driver packages, runs Unit + Functional suites, and publishes results/artifacts.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| eng/pipelines/ci/unit/sqlclient-ci-unit-stages.yml | Defines the Windows/Linux/macOS runtime/SNI job matrix for the unit+functional test pipeline. |
| eng/pipelines/ci/unit/sqlclient-ci-unit-pipeline.yml | New pipeline entrypoint that is triggered by sqlclient-ci-package completion and invokes the stage matrix. |
| eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml | Job template to align to upstream commit, consume produced packages, run Unit/Functional tests, and publish results. |
paulmedynski
added
the
Area\Engineering
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:100
- The comment says this step downloads the exact driver packages, but
sqlServerVersionOverride: 1.0.0means$(sqlServerPackageVersion)will be pinned instead of using the version produced by the triggeringsqlclient-ci-packagerun. That makes the current comment misleading and obscures why the override exists.
Update the comment to reflect the pin (or remove the override if the intent is to validate the exact SqlServer package from the upstream run).
# Download the exact driver packages produced by the triggering pipeline.
- template: /eng/pipelines/common/steps/download-driver-packages-step.yml@self
parameters:
sqlServerVersionOverride: 1.0.0
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:120
update-config-file-step.ymlis being used only to setUseManagedSNIOnWindows, but because most parameters are omitted (so they default to empty/false), this step also overwrites the non-empty defaults inconfig.default.jsonc(e.g., TCP/NP connection strings andSupportsIntegratedSecurity=true). That unintentionally changes the baseline test config for this pipeline.
Pass through the defaults from config.default.jsonc so this step doesn’t clobber them while toggling SNI.
# Configure the test suite's Windows SNI implementation.
- template: /eng/pipelines/common/templates/steps/update-config-file-step.yml@self
parameters:
debug: ${{ parameters.debug }}
saPassword: ''
UseManagedSNIOnWindows: ${{ parameters.useManagedSNI }}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (1)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:8
- The header comment claims this job runs against the exact packages from the triggering
sqlclient-ci-packagerun, but the job forcessqlServerPackageVersionviasqlServerVersionOverride: 1.0.0, soMicrosoft.SqlServer.Serveris not necessarily taken from the upstream artifact (NU1605 workaround). Please adjust the comment to reflect this exception to avoid misleading future maintainers.
# Builds and runs the SqlClient Unit and Functional test suites in Package reference mode against
# the exact packages produced by the triggering sqlclient-ci-package run.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:107
- This job sets sqlServerVersionOverride: 1.0.0 while the adjacent comment says it downloads the “exact driver packages”. Since the override intentionally diverges from the artifact-resolved version (to avoid NU1605 downgrade warnings), add an in-file explanation (similar to managed-instance/stress jobs) so the behavior is clear.
# Download the exact driver packages produced by the triggering pipeline.
- template: /eng/pipelines/common/steps/download-driver-packages-step.yml@self
parameters:
sqlServerVersionOverride: 1.0.0
eng/pipelines/ci/unit/sqlclient-ci-unit-job.yml:7
- The header comment says this job runs against the “exact packages produced” by the upstream run, but this job pins Microsoft.SqlServer.Server via sqlServerVersionOverride (so at least that package version is not taken from the upstream artifact). Update the comment to reflect the exception so future readers aren’t misled.
This issue also appears on line 104 of the same file.
# Builds and runs the SqlClient Unit and Functional test suites in Package reference mode against
# the exact packages produced by the triggering sqlclient-ci-package run.
| -p:PackageVersionSqlServer=$(sqlServerPackageVersion) | ||
| -p:TestResultsFolderPath=TestResults | ||
|
|
||
| - task: PublishTestResults@2 |
There was a problem hiding this comment.
Using the common template for publishing test results.
|
|
||
| steps: | ||
|
|
||
| # Align the checkout with the commit that produced the upstream packages. |
There was a problem hiding this comment.
I'm starting to see some common boilerplate in these jobs, so we will keep an eye out for some common steps to extract, but not yet.
| ${{ else }}: | ||
| testResultsFiles: | | ||
| TestResults/*.trx | ||
| TestResults/**/*.coverage |
There was a problem hiding this comment.
This was causing PublishTestResults@2 to silently fail when it encountered non-TRX files. Code coverage publishing is an entirely different subject we will tackle later.
| - Windows | ||
|
|
||
| # Optional test run title. When empty, preserve the historical OS-based title. | ||
| - name: testRunTitle |
There was a problem hiding this comment.
The old way would publish multiple test runs with the same titles like "Windows Tests". Now, the new CI pipelines use the job name (like "Win : Native SNI : net9.0" and "macos : net10.0") which is much more descriptive.
paulmedynski
assigned
mdaigle and
apoorvdeshmukh and unassigned
benrr101 and
cheenamalhotra
mdaigle
left a comment
mdaigle
left a comment
There was a problem hiding this comment.
Everything in here looks correct. But I don't understand how this is fitting into the overall testing strategy. Isn't all of this already covered by the regular CI pipeline? What additional coverage do we get from running unit tests in package mode?
|
@mdaigle - Sent you the new CI pipeline strategy directly. The legacy CI pipeline will be sunset once all of the new CI pipelines are up and running. I don't expect this particular pipeline to add coverage over-and-above what |
|
Moved back to Draft until its stack parent is merged since there will be conflicts to resolve. |
The signed sqlclient-ci-package driver assemblies grant InternalsVisibleTo to test assemblies signed with the dedicated test key's public key. Download the sqlclient-test-key.snk secure file and expose it as TestSigningKeyPath so build.proj signs the unit-test assemblies accordingly.
Promote the target-framework loop from the managed-instance job up into the stage, so each OS/SNI x runtime combination runs as its own parallel job with a single Unit/Functional/Manual test pass (job/display names now include the TFM). Add a sqlServerVersionOverride parameter to the shared download-driver-packages step and pass 1.0.0 from the managed-instance and stress jobs, so restore uses the released stable Microsoft.SqlServer.Server instead of the -ci prerelease and avoids the NU1605 downgrade against Microsoft.SqlServer.Types' >= 1.0.0 dependency. Overall package versioning is being addressed separately.
afe29fe to
8804319
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.
Suppressed comments (1)
eng/pipelines/ci/managed-instance/sqlclient-ci-managed-instance-stage.yml:10
- This template file appears to be unused in the repo (no pipeline/templates reference
sqlclient-ci-managed-instance-stage.yml), whilesqlclient-ci-managed-instance-pipeline.ymlcurrently usessqlclient-ci-managed-instance-stages.yml. Keeping an unreferenced alternative stage template is confusing and adds maintenance surface; either remove this file or update the managed-instance pipeline to consume it (and delete the older stages template if it becomes redundant).
# This stage builds and runs the SqlClient Unit, Functional, and Manual test suites against an Azure
# SQL Managed Instance, building the test projects in "Package" mode against the NuGet packages
# produced by the sqlclient-ci-package pipeline.
#
# It fans out to one job per OS:
|
/azp run |
|
Azure Pipelines: Successfully started running 3 pipeline(s). |
Description
Adds a package-triggered
sqlclient-ci-unitpipeline for the SqlClient Unit and Functional test suites.I decided not to bother adding separate pipelines for unit, functional, and simulated tests right now. We can decide if such a distinction is worth it later.
net462on Windows with native SNI.net8.0,net9.0, andnet10.0on Windows with native and managed SNI.net8.0,net9.0, andnet10.0on Linux and macOS with managed SNI.ADO-Win25,ADO-UB24, and the Microsoft-hostedmacos-latestimage.sqlclient-ci-packagerun.build.projdefault filter.Testing